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ABSTRACT 

This paper exaunines the evaluation of student 
performance (SP) in schools in order to develop a set of strategies 
for creating more responsive systems for its evaluation, focusing on 
disadvantaged students. Problems involved in the evaluation of the 
performance of disadvantaged students are examined, including: 
evaluation purposes (direction, moti at ion, certification, and 
selection) that are in conflict, task assignments that are overly 
routine, criteria that are unclear and narrow, standards that deny 
disadvantaged students the feeling of competence, testing practices 
that provide limited information or SP, appraisals that limit teacher 
discretion, feedback, that is too limited, and the absence of planning 
for the improvement of SP. Elements of a more responsive evaluation 
system for at-ris)t students include: attention to the purposes of 
direction and motivation; consideration of ta^Ks prior to developing 
the evaluation process; broad criteria tapping multiple abilities, 
processes, and non-academic domains; standards that are challenging 
yet attainable; a broader range of information and more varied 
collection techniques; more efficient methods of collecting 
information on SP; appraisals that involve teacher discretion and 
broader teacher participation; more detailed feedback; new ways of 
presenting certification/selection information to students and new 
ways of presenting direction/motivation to others; and improvement 
plans that offer opportunities and resources. A 59-item list of 
references is included. (RLC) 
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The Center 



The mission of 'he Center for Research on Effective Schooling for Disadvantaged Students 
(CDS) is to significantly improve the education of disadvantaged students at each level of 
schooling through new knowledge and practices produced by thorough scientific study and 
evaluation. The Center conducts its research in four program areas: The Early and Elementary 
Education Program, The Middle Grades and High Schools Program, the Language Minority 
Program, and the School, Family, and Community Connections Program. 

The Early and Elementary Education Program 

This program is working to develop, evaluate, and disseminate instructional programs 
capable of bringing disadvantaged students to high levels of achievement, panicularly in the 
fundamental areas of reading, writing, and mathematics. The goal is to expand the range of 
effective alternatives which schools may use under Chapter 1 and other compensatory education 
funding and to study issues of direct relevance to federal, state, and local policy on education of 
disadvantaged students. 

The Middle Grades and High Schools Program 

This program is conducting research syntheses, survey analyses, and field studies in middle 
and high schools. The three types of projects move from basic research to useful practice. 
Syntheses compile and analyze existing knowledge about effective education of disadvantaged 
students. Survey analyses identify and describe current programs, practices, and trends in middle 
and high schools, and allow studies of their effects. Field studies arc conducted in collaboration 
with school staffs to develop and evaluate effective programs and practices. 

The Language Minority Program 

This program represents a collaborative effort. The University of California at Santa 
Barbara is focusing on the education of Mexican-American students in California and Texas; 
studies of dropout among children of recent immigrants arc being conducted in San Diego and 
Miami by Johns Hopkins, and evaluations of learning strategics in schools serving Navajo, 
Cherokee, and Lumbce Indians are being conducted by the University of Northern Arizona. The 
goal of the program is to identify, develop, and evaluate effective programs for disadvantaged 
Hispanic, American Indian, Southeast Asian, and other language minority children. 

The School, Family, and Community Connections Program 

This program is focusing on the key connections between schools and families and between 
schools and communities to build better educational programs for disadvantaged children and 
youth. Initial work is seeking to provide a research base concerning the most effective ways for 
schools to interact with and assist parents of disadvantaged students and interact with the 
community to produce effective community involvement. 



Abstract 



This paper examines the evaluation of student perfonnance in schools in order to develop a 
set of strategies for creating more responsive systems for the evaluation of student performance. 
The paper examines problems involved in evaluation of the performance of disadvantaged 
students, including problems in the purposes of evaluation, task assignment, criteria, standards, 
testing, appraisal, feedback, and planning for improvement. A set of guidelines is derived for 
creating evaluation systems that would be more responsive to disadvantaged students. 



Introduction 



This paper examines the evaluation of student per- 
formance in schools in order to develop a set of 
strategies for creating more responsive systems for 
the evaluation of student perfonnance. The concept 
of responsive evaluation systems derives from a 
more general concept of responsive schools. Re- 
sponsive schools are those in which student per- 
formance and participation are met with timely and 
appropriate reactions from adults (Natriello, 
McDilU and Pallas, 1990; McParUand and McDiU, 
1977). In responsive schools, staff recognize and 
reward good student performance and identify and 
penalize poor student performance. Moreover, 
school staff identify the larger causes of poor stu- 
dent performance and develop remedies^ both inter- 
personal and stnicturaU to create conditions under 
which students can succeed. 

Evaluation processes are key elements in schools 
(Natriello, 1987). Student performance and behav- 
ior are routinely monitored and evaluated. Indeed, 
many of the activities of teachers, from formal 
testing to infomial observations and comments, can 
be seen as evaluation of student performance in the 
context of the instructional and extracurricular pro- 



gram of the school. It is through evaluations that 
the school attempts to influence the development of 
students. Thus, evaluations and student evaluation 
systems are a central element in any auempi to crc • 
ate more responsive school environments. 

The process of evaluating student performance in 
schools and classrooms is complex and involves a 
variety of activities. Each of these elements may 
present problems for disadvantaged students. 
Evaluation systems are often characterized by (1) 
purposes that are in conflict, (2) tasks that tend to 
be overly routine, (3) criteria that are unclear and 
narrow, (4) standards that deny disadvantaged 
students the feeling of competence, (S) tests that 
provide limited information on student perfor- 
mance, (6) appraisals that limit teacher discretion, 
(7) feedback that is too limited, and (8) the absence 
of plans for the improvement of student perfor- 
mance. We consider these and other problems in 
evaluation systems as a prelude to developing a 
series of guidelines for creating evaluation systems 
that would t)e more responsive to disadvantaged 
students. 



Problems in Evaluation Systems for Disadvantaged Students 



Purposes In Conflict 

There are at least four purposes for student evalua- 
tion systems: direction, motivation, certiFication, 
and selection. Direction refers to the goal of pro - 
viding students with clear and specific infonnation 
on their performance and how to modify it along 
the lines desired in the school. Motivation involves 
securing the cooperation and commitment of those 
being evaluated to the tasks at hand. Both of these 
purposes are compatible with the development of 
responsive evaluation systems. Evaluation activi- 
ties oriented to these purposes can lead to school 
and classroom practices that respond to the needs 
and perfonnance problems of all students, particu- 
lariy disadvantaged students, who often lack access 
to external sources of motivation and direction. 

However, the other two purposes for evaluation 
systems, certification and selection, do not neces* 
sarily lead to activities that respond to student 
needs. Certification and selection respond to t^c 
needs of others, not to the needs of students. Cer- 
tification provides assurance to others that a student 



has attained a certain level of accomplishment or 
mastery. Selection is the identification of some 
students for future educational or occupational op- 
portunities. Although certification and selection of- 
fer certain benefits to those students who arc ccrii - 
fied and/or selected, they lead to practices that 
contribute little to the development of students. 

The tension between the purposes of certification 
and selection and those of direction and motivation 
are only apparent in the case of the evaluation of 
youngsters who have trouble succeeding in school 
Students with reasonable chances of success profit 
from the certification and selection functions of 
evaluations as well as from the direction and moti - 
vation functions. For disadvantaged students, who 
are unlikely to experience success as a result ol 
those practices set in motion by the certification and 
selection fiincMons of evaluation, the direction and 
motivation functions become even more important. 
However, the basic tension between direction and 
motivation, on the one hand, and certification and 
selection, on the other, creates problems in design - 
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ing all aspects of evaluation processes for 
disadvantaged students. 

Assigning Tasks: Between Boredom and 
Anxiety 

Evaluation processes set up situations in which all 
students are at risk of failure. The level of risk 
varies with the nature of the tasks on which stu - 
dents are being evaluated. Dombusch and Scott 
(1975:80) have suggested that tasks differ in pre- 
dictability, that is, "the extent to which the per- 
former has knowledge of which path is most likely 
to lead to success." Furthermore, they demon- 
strated that when tasks are predictable, performers 
prefer that they be given the specific procedures to 
be followed to execute the tasks, but that w!ien 
tasks are less predictable, performers prefer the 
autonomy to make their own decisions about how 
to perform the task. 

Predictable tasks accompanied by specific 
procedures to be followed cany relatively less risk 
than less predictable tasks. In an analysis of 
classroom tasks £>oyle (1983) classified opinion 
tasks and certain memory tasks (i.e., those 
involving the reproduction of small amounts of 
material) and certain routine tasks (i.e., those re- 
quiring relatively simple algorithms) as low in risk, 
and undemanding tasks and other memory tasks 
(i.e., those involving the reproduction of large 
amounts of material) and other routine tasks (i.e., 
those involving complicated procedures) as high in 
risk. 

Research in classrooms has revealed two forces at 
woric to reduce the risk associated with the evalua- 
tion process. First, Davis and McNight (1976) and 
Wilson (1976) showed that students resist attempts 
to shift from low risk routine or procedural tasks to 
high risk understanding tasks in classrooms. 
Moreover, students also attempt to renegotiate as- 
signed tasks so that they are more predictable and 
entail less risk. 

Second, there appears to be a tendency to structure 
evaluation activities as if the tasks being evaluated 
are predictable. For example, Armbruster, 
Stevens, and Rosenshine (1977), in a study of 
three reading curricula, found that although the 
texts emphasized comprehension and interpretation 
skills, the tests solicited factual information from 
students based on the ability to locate infonnation 
in the text. Bailey, Brazee, Chiavaroli, Heiteck, 
Lechner. Lewis, McKittrick, Redwine, Rcid, 
Robinson, and Spear (1988) found that despite an 
intention to view reading and writing as meaning - 
centered processes, the evaluation system tended to 
encourage practice in isolated drill and skill 



exercises. To counteract this it was necessary to 
create evaluation and reponing instruments which 
monitored the extent to which smdents took risks as 
readers and writers to engage in learning tasks 
consistent with the meaning-centered approach. 

Studies of typical classrooms that reveal a tendency 
to avoid less predictable tasks and utilize routine 
tasks to minimize the risk associated with evalua- 
tion processes suggest that these tendencies may be 
even more severe in classes serving disadvantaged 
youngsters. Such youngsters are even more likely 
to experience failure under traditional evaluation 
processes and so face even greater risks when con - 
fronting less routine tasks. However, the retreat 
fhxn unpredictable tasks to minimize risk and anx- 
iety may leave disadvantaged students in class- 
rooms where the instructional tasks arc insuffi- 
ciently interesting to hold their attention (Oakes, 
1985). 

Criteria: Focusing Attention or Narrowing 
Opportunity 

Criteria are those properties of tasks that are taken 
into account when making an evaluation of perfor- 
mance. For example, in assessing student perfor- 
mance in writing, teachers may apply criteria re- 
garding creativity and imagination as well as criteria 
pertaining to correct grammar and proper usage. 

Several issues surrounding the development and 
use of perfonnance criteria have impoitant implica - 
tions for the evaluation of disadvantaged students. 
Studies of the impact of evaluation processes on 
students have revealed that criteria are often not 
cleariy communicated to students (Natriello, 1982) 
and that criteria are often not linked to the instruc- 
tional program (Linn, 1983; Rudman, Kelly. 
Wanous, Mehrens, Qaric, and Porter, 1980). Each 
of these problems presents difftculiies for all stu- 
dents, but the difficulties may be worse for disad- 
vantaged students, who lack family resources to 
help them understand the demands of their teachers 
and direct their effons accordingly. 

Efforts to clearly articulate criteria and link them to 
the instructional process have the potential for 
improving the evaluation process for disadvantaged 
youth by focusing attention on the most important 
aspects of the curriculum. If these efforts at greater 
articulation and alignment result in a narrowing of 
the criteria and a specification of tasks to the point 
where routine tasks dominate the classroom, 
however, they also have the potential to create 
additional problems for disadvantaged youth. The 
narrowing of criteria for the evaluation of 
performance results in a reduction of opportunities 
to experience success in the classroom. For 
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disadvantaged students, who typically have dif- 
ficulty with academic worV, a reduction in oppor- 
tunities to succeed may have serious negative ef- 
fects on motivation. Moreover, if classroom tasks 
become overly routinized, students may perceive 
them as boring and uninteresting. There are prac- 
tices and special effons to broaden the base of cri • 
teria for evaluations. As Thomdike ( 1%7) notes: 

In practice, certainly many other 
considerations than that of pure competence do 
enter into maiks. Such factors enter in as ( 1 ) 
industry and effon - i.e., completing all 
assigned work and even doing optional work 
for "extra credit" (a kind of educational bribe); 
(2) frequent and active participation in class 
discussion; (3) neatness in written woric and 
mechanical correctness in such areas as 
spelling and grammar; and (4) personal 
agreeableness, attractiveness, cleanliness, and 
docility. To some extent and by some 
instnictors. certain of these factors would be 
endorsed as legitimate influences on a mark. 
Others would more uniformly be accepted as 
extraneous influences to be minimized as far 
as possible, (p. 762) 

Holmes ( 1978) notes that other criteria which make 
it possible for an organization to operate often 
covertly enter into the evaluation process. These 
include criteria related to behavior and effort such 
as politeness, conformity, and perseverance. Sal- 
ganik (1982) reports on a system in which students 
were evaluated on three criteria - achievement, ef- 
fort, and conduct. In a national survey of sec- 
ondary school teachers, Natriello and McPartland 
(1987) found that smdem effort was "veiy impor- 
tant" or "extremely important" in the evaluation 
process of over 70% of the teachers. 

Many of the strategies for broadening the set of 
criteria upon which evaluations are based involve 
including criteria of a more formative nature (i.e., 
those factors, such as effort and perseverance, that 
contribute to ultimate achievement) as well as sum - 
mative factors. Rudman (1987) discusses this as a 
strategy of linking assessmem with teaching and 
Broadfooc (1986) sees it as attending to the curricu- 
lum function of reporting systems as well as to the 
communication functions. Paikay (1987) provides 
some initial evidence that such approaches may be 
particulariy effective for disadvanuged students. 
He presents evidence indicating that students in re- 
medial English classes in an urban high school 
demonstrated greater knowledge and more favor- 
able attitudes toward the subject matter after being 
graded for a 3-week period on just the completion 
of assignments instead of traditional grading proce- 



dures which take into account the quality of the 
work completed. 

Broadening the base of criteria upon which students 
are evaluated off rs increased (^portunities for dis- 
advantaged youngsteis to succeed in schools. Of 
course, one strategy is simply expanding the range 
of the academic performance criteria. One approach 
to providing sucli opportunities for success for stu- 
dents who would otherwise experience consistent 
failure in the classroom involves a restructuring of 
the tasks of the classroom so that they draw on a 
wider range of ability dimensions. Such multiple - 
ability classrooms (Cohen, 1986; Rosenholtz, 
1977) attempt to move beyond the narrow range of 
academic tasks and criteria, most of which rely 
upon reading skills, so that every student can expe- 
rience some success. In the multiple-ability class- 
room, the intention is for all students to find some 
task at which they can experience a sense of 
oxnpetenoe. 

Standards: Between A Rock and A Hard 
Place 

Criteria arc those dimensions of tasks upon which 
evaluations wiU be based. Standards are the levels 
of performance expected on the criteria. Research 
and commentary on appropriate standards for the 
evaluation of student performance have focused on 
three types of standards: those set in refe^'ence to 
the criterion level of a group, those set in reference 
to some absolute criterion level, and those set in 
reference to the previous criterion level of an indi- 
vidual (Wise and Newman. 1975; Rheinberg, 
1983; Thomdike, 1969). Each of these types of 
standards has different implications for the evalua- 
tion of disadvantaged students. 

Terwilliger(1977) links mrm-referenced standards 
to what he tenns the pragmatic [rfiilosophy, a view- 
point primarily concern^ with practical choices and 
the consequences of such choices. An evaluation 
system which differentiates among individual stu- 
dents is optimal for identifying the choices available 
to those who make decisions about admission to 
future educational and employment opportunities. 
Thus, nonn-referenced standards serve the purpose 
of selection identified eaiiier. 

Norm-referenced standards have been criticized in 
many quarters. Bresee (1976) lists a series of 
proUems with such standards: 

(1) the conflict between the necessity of producing 
a normal distribution of grades and the goal of 
having teachers produce improvement in all stu- 
dents in a class; 



(2) the distonion of the curriculum as teachers seek 
to diversify instructional objectives to produce a 
range of achievement in a class; 

(3) the diversion of student attention from the task 
at hand to the perfonnance of other students; aiK) 

(4) the introduction of false competition because 
achievement is not really in limited supply. 

To these i:>eutsch (1979) adds: (1) the distortion of 
the testing process so that tests take the fonn of 
contests in which all perfonners participate under 
uniform conditions: (2) the lack of rewards created 
by the artificial scarcity of good grades that is likely 
to impede the development of students' sense of 
their own value; and (3) the encouragement of 
competition which may be counterproductive for 
tasks requiring cooperation and communication. 
Much of the criticism of norm-referenced standards 
has to do with their being ill-suited to the purposes 
of certification (Glaser. 1963), direction and moti- 
vation (Beady and Slavin, 1981). 

Absolute or criterion-referenced standards have 
enjoyed increased support in recent years. They are 
based on some externally determined level of per- 
fonnance. TerwUliger (1977) associates such stan- 
dards with the behaviorist perq>ective. The actual 
strategy for setting such absolute standards has 
been challenged by Glass (1978) and Burton 
(1978), who conclude that they must be set arbi- 
trarily, and by Shepard (1976), who concludes that 
all cuncnt methods of setting absolute standards re- 
duce to some fonn of nonn-referenced standards. 
Criterion-referenced standards may serve the pur- 
pose of certification, but they often fail to provide 
motivation and direction to students, paiticulariy 
when decisions about standards are removed from 
the informed professional opinion of teachers 
(Burton, 1978). 

Individually-referenced or self-referenced starxiards 
are based on comparing a student's current perfor- 
mance with some other feature of the student. 
Terwilliger (1977) points out that current perfor- 
mance may be compared with eariier performance 
or with a student's ability. Terwilliger ( 1977) as- 
sociates self-referenced standards with the humanist 
view of education, and sees them as a strategy to 
recognize individual differences, reward effon, and 
foster motivatioa 

Disadvantaged students are likely to have limited 
opportunities to experience success under nonn- 
referenced or criterion-referenced standards. 
Systems based at least in part on self-referenced 
standards would offer disadvantaged students some 
sense of competence. The dilemma, of course, is 



that systems that rely solely upon self-referenced 
standards deny students information on how their 
perfonnance may be viewed by a broader audience. 
Such systems may fail to challenge students and 
leave them with misconceptions about their own 
performance (Natriello and Dombusch, 1984; 
McDill, Natriello. and Pallas. 1985). 



Collecting Information: Tests and Contests 

Collecting information on student performance for 
the purposes of evaluation is a demanding and time- 
consuming task. Because it is impractical if not 
impossible to collect complete information on stu- 
dent performance, the collection of such informa- 
ti(m involves samiking decisions. The challenge is 
to insure that the infonnation collected provides a 
valid and reliable estimate of performance appro- 
priate to the purposes, tasks, criteria, and standards 
of the evaluation process. A number of analysts 
have contributed important observations about the 
relationship between the dominant method of col- 
lecting perfonnance infonnation - testing ~ and the 
purposes, tasks, criteria, and standards for evalua- 
tion. Deutsch (1979) argues that the structure of 
most testing situations is a reflection of the prevail- 
ing purpose of evaluation (selection) and the types 
of standards utilized (norm-referenced). He notes 
that: 

The social context of most educational 
measurement is that of a contest in which 
students are measured primarily in comparison 
with one another rather than in terms of 
objective accomplishment. If educational 
measurement is not mainly in the fonn of a 
contest, why are students often asked to reveal 
their knowledge and skills in carefully 
regulated test situations designed to be as 
uniform as possible in time, atmosphere, and 
conditions for all sbidents? Individuals vary 
enormously in terms of the amount of time 
they need and the kind of atmosphere and 
circumstances thLi facilitate or hinder their 
expression of their knowledge and skills; it is 
only the comparison of students with one 
aixHher that requires measures of educational 
achievement that take the form of contests 
(Deutsch, 1979: 394). 

As long as testing is constructed as a contest with a 
limited number of winners, disadvanuged students 
are likely to turn out the losers. Deutsch goes on to 
describe the damaging effects of norm-referenced 
standards for individual students and advocates an 
evaluation system that would provide individual- 
ized, particularistic feedback to students to foster 
their development. Thus, his objection to the typi- 
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cal testing situation is rooted in a rejecticm of evalu- 
ation for the purpose of selection and of standards 
that are noim-referenced. Of course, both selection 
and norm-referenced standards are likely to lead lo 
practices inimical to disadvantaged students. 

Although much of what has been written on the 
collection of information on student perfoimance 
has centered on tests and testing, there is a range of 
alternative methods of collecting such infonnation. 
For example, Gaston (1976) suggests monitoring 
students' unassigned reading in the library and lis- 
tening to student conversations as they leave the 
classroom as a way to collect information on stu- 
dent attitudes and behavior. Heller ( 1978) sug- 
gests alternatives to standardized reading tests such 
as the use of reading materials from popular 
magazines, fables, and poems. Solo ( 1977) ex- 
plains how alternatives such as anecdotal records 
aiKl collections of students' daily work may be used 
to provide insight into student performance. Her- 
man and Dorr- Bremme ( 1984) note a variety of 
techniques used by teachers to collect information 
on student performance, including routine class and 
homework assignments, classroom interaction 
duri.ng question and answer sessions, recitations, 
discussions, oral reading, problem-solving at the 
chalkboard, special projects, presentations, and re- 
ports. 

Such alternative methods of collecting information 
on performance may provide disadvantaged 
students with a wider variety of ways to 
demonstrate their competence. At the very least, 
they provide those who work with disadvantaged 
students with a broader base of information for 
understanding their academic performance and 
needs. 

Appraisals: Sensitivity versus Bias 

Appraisal involves comparing the information col- 
lected on student performance on assigned tasks 
with the criteria and standards previously estab- 
lished for those tasks. But appraisal is a more 
complex process than this description would sug- 
gest. As Dombusch and Scott (1975) observe: 

The application of standards in specific 
situations is rarely a simple or straightforward 
procedure. It requires judgment with respect 
to the comparability of the performance 
situation and the situations for which the 
standards are considered applicable ... 
appraisal is seldom a mechanicfu procedure. 
In short, appraisal entails deciding how to 
interpret a low or high performance score. 
Accurately appraising a task performance 
requires knowledge of extenuating 
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circumstances, whether it be the inexperience 
of the task performer, the lack of facilities, or 
assistance received from a more skilled co- 
worker. Such information is of critical 
importance in determining what« if any, 
message is to be communicated to the 
perfomier concerning the quality of his or her 
task performance (p. 143). 

But the exercise of discretion that is so critical to 
producing an appropriate appraisal is viewed as 
dangerous in ihc case of teachers appraising the 
perfomiance of students (Archer and McCarthy, 
1988), particularly disadvantaged students. The 
fear, of course, is that teachers will be biased 
against disadvantaged students either by failing to 
recognize their strengths (e.g., Rosenthal and Ja- 
cobson, 1968; Rist, 1970), or failing to challenge 
them to perform to their capacity (Natricllo and 
Dombusch, 1984), or both. 

This fear has led teachers and those who supervise 
them to seek to reduce teacher discretion in the 
evaluation process. This can be done through 
increased used of standardized tests and through 
teacher grading practices that arc more formulaic 
and allow less room for judgment. These practices 
are seen as easier to defend, if they are challenged 
(Natriello, 1982). However, the dilemma is that 
such practices limit the exercise of teacher 
discretion which may be important to developing 
evaluations that provide sufficient direction and 
motivation to disadvantaged students. Teachers, 
those individuals in the best position to be more 
sensitive to students, their performance, and their 
condition, may be overly constrained by fears of 
possible bias. Of course, neither teacher bias nor 
constrained teacher discretion is beneficial to 
disadvantaged students. 

Full teacher discretion without bias represents the 
best condition for appraisal of the work of all stu- 
dents, including disadvantaged students. One strat- 
egy for achieving such conditions might be to 
broaden the production of and the audience for 
teacher appraisals. Currently, most teacher ap* 
praisals of student performance are the product of a 
single teacher and are meant for a single student and 
peitiaps his or her parents. But it is possible to in- 
volve more than one teacher in the appraisal process 
or for teachers to share their appraisals with each 
other. These processes would preserve and per- 
haps even increase teacher discretion while provide 
ing checks and balances on the possible biases of 
any single teacher. To encourage teachers to work 
together on the appraisal of student performance it 
will be necessary to overcome teachers' habits of 
not relying on information from other teachers 
(E>orr'Br:mme, 1983). 
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Feedback: The Economies of Providing 
Information 

Providing feedback on perfomiance evaluations is 
an impoitant component of the evaluation process. 
If the results of evaluations are never communi- 
cated, there is little reason to expect them to have an 
effect The results of evaluations may be commu- 
nicated to multiple audiences, including the student, 
parents, school officials, and potential employers 
(Ahmann and Clock, 1967). 

The feedback process can be quite varied and con- 
tribute to the multiple purposes of evaluations. 
Slavin (1978) distinguishes three types of feedback 
regarding student perfonnance: feedbiiick that tells 
students where they stand compared m other stu- 
dents, based on nonn-referenced standards; feed- 
back that provides students with infoimation on 
Uieir on-going perfonnance, based on ciiterion-ref- 
erenced standairds; and feedback that enluuices stu- 
dent motivation, based on timely assessments on 
tasks that arc neither too difficult nor too easy. 
Lissman and Paetziod ( 1983) distinguish between 
infbmiative feedback and motivational feedback. 

Feedback creates several dilemmas for the evalua- 
tion of disadvanuged students. First, different 
kinds of feedback will often result in quite different 
effects on students. For example, disadvantaged 
students may be achieving quite pooriy in terns of 
norms for the broader population of students, but 
tiiey may be achieving quite well in temis of their 
previous perforaiance. If students receive infomia- 
tion on their perfomiance in relation to the norms 
they may be disheartened. If they receive informa- 
tion on Uieir performance in relation to their previ- 
ous performance, they may be misled about their 
opportunities in tbc broader society. If they receive 
b(^ kinds of feedback they may be quite conlused 
as to where they actually stand. Thus, providing 
the full range of feedback to disadvantaged students 
is a complex undertaking. 

Second, providing feedback is a time-consuming 
process tiiat is often made manageable by the re- 
duction of information on student performance to 
ccHnmon symbols such as grades. Although these 
symbols may serve the needs of external audiences 
quite well (e.g., prospective employen), they often 
do not serve the needs of students for detailed in- 
fomiation on perfonnanoe problems to shape future 
efforts to achieve. Moreover, shorthand symbols 
often do not serve the needs of school officials, 
who must match student needs with available 
school resources. A grade of "D" given to 2 
students may result from very different academic 
problems requiring different remedies, yet the 
single grade fails to communicate a clear and 



detailed meaning tied to a range of possible 
responses. 

Detailed feedback seems to increase students' 
motivation atKl performance. BuUer and Nisan 
(1986) found that sixth graders scored higher on 
both a quantitative task and a qualitative (diverigent 
thinking) task when they received task-related 
comments on their performance. In contrast, 
students receiving ^ades scored high on the 
quantitative task uid low on the qualitative task, 
and students receiving no feedback scored low on 
both tasks. Moreover, snidents receiving written 
comments expressed greater interest in tiie tasks 
Uian students in the other two groups. 

There may be particular advantages to using de- 
tailed feedback with disadvantaged students. Maas- 
de Brouwer ( 1986) reports on several studies indi- 
cating that initially poor performing students profit 
more from detailed feedback and criterion-centered 
feedback, and that tiiese effects occur in the aca- 
demic, social, and affective domains. 

The dilemma is that providing detailed feedback re - 
quires a great deal of time, yet detailed feedback 
provided shortiy after tiie performance being evalu- 
ated is what students, particulariy disadvantaged 
students, require to plan future efforts accordingly. 
A number of investigators are developing alterna- 
tives to traditional feedback processes. Rudman 
(1976) advocates the use of reporting device? such 
as checklists that are more closely related to tiie 
mechanisms for recording student perfonnance. 
Such medianisms may allow greater efficiency in 
the feedback process. Ediger (1975) suggests more 
frequent and more varied mechanisms for reporting 
students' performance, including telephone and 
face-to-face conferences with parents. Giannangelo 
and Lee (1974) and Giannangelo (1975) describe a 
system of computer-asasted reporting that provides 
more anecdotal information on student 
performance. Holtz ( 1976) presents a r^ning 
method for students' performance in elementary 
science more cleariy related to evaluation criteria. 

Natilello (1982) reports on non-traditional metiiods 
such as the use of cassette tapes of teachers' 
comments, open gradebooks that students can 
examine at any time, and procedures for involving 
students in taUying their own cumulative scores at 
various points in the grading period. Walling ( 
1975) discusses five broad categories of re porting 
technicjues ~ traditional grades, percentage ratings, 
checklists of objectives, narrative evaluations, and 
conferences. Stewart (1975) describes a multi- 
dimensional reporting system for use in elementary 
schools. 
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Investigatois and practitioners are also considering 
the use of student profiles or portfolios, composites 
of various kinds of information on student perfor- 
mance (Bank & WiUiams. 1987; Mayfield, i983: 
Phoenix Union High School District, 1981). With- 
ers (1986) presents a typology of the various kinds 
of profiles and Broadroot ( 1986) describes one 
such profile, the record of achievement in use in 
England and Wales. 

Planning for Improvement 

In temis of improving student perfonnance, evalua- 
tions of student perfonnance that end with the 
communication of the evaluation may have little im - 
pact, though such evaluation processes may serve 
the purposes of certification and selection. Simply 
communicating an evaluation is unlikely to provide 
adequate direction and motivation for students. 
Thus, it is important to link the evaluation process 
with efforts to plan and deliver instructional or 
other services that can help students, especijdly dis- 
advantaged students, to succeed in school. 

In particular, evaluative information should enable 
schools to identify student needs more accurately 
and to assign school resources - instiiiction, coun- 



seling, books and materials, and so on - where and 
when they are needed by students. Unfortunately, 
most schools are not equipped to respond quickly 
and ^jpropriately on the basis of information about 
student perfonnance. Student assessment informa- 
tion is used in school plarm'ng efforts only in C\e 
most rudimentary ways - for example, in 
infrequent decisions about student course 
placement. Even teachers, who woric most direcUy 
with students, base their plarming decisions on 
factors otiier than student assessments (Clark & 
Peterson, 1986; Shavelson & Stem, 1981; 
Stiggins, ConkJin. & Bridgeford, 1986). 

Often, school staff plan on the basis of uends in tiic 
assessments of previous cohorts of students, 
because they cannot obtain and process current 
assessment information quickly enough. When 
teachers and other school staff'^ do not act on the 
basis of student assessment information, students 
and their parents must assume the major burden of 
developing appropriate responses to evaluative 
information. This is especially problematic for 
disadvantaged students who lack the personal, 
family, and community resources to interpret 
evaluations and react appropriately to them. 



Elements of a More Responsive Evaluation System for At-Risk Students 



The literature on the evaluation of students in 
schools and classnx)ms provides a number of in- 
sights for the design of evaluation systems for at- 
risk students. The same systems that are designed 
with disadvantaged students in mind are also likely 
to be appropriate for all students. These systems 
would incorporate the following elements. 

1. Attention to the Purposes of Direction 
and Motivation 

Because evaluation systems typically encompass 
multiple purposes, it is useful to consider each of 
the purposes in the design of Uie system. Because 
evaluation systems have most often neglected the 
purposes of direction and motivation, it is particu- 
lariy important to consider these purposes when 
designing evaluation systems for disadvantaged 
students. This can be done by considering each of 
the elements of the system in terms of these pur- 
poses. 

2. Consideration of Tasks Prior to Devel- 
oping the Evaluation Process 

Existing evidence indicates Uiat evaluation pro- 
cesses often lead to the development or interpreta- 



tion of tasks in ways that minimize the risk experi- 
enced by those being evaluated by resorting to rou- 
tinization. Such routinization, while offering some 
protection to students from evaluation processes 
ur.der conditions of uncertainty, also leads to less 
motivating school experiences. These trends arc 
even more pronounced in classes and program .s 
serving disadvantaged shJdents. 

To avoid such a trend it is necessary to consider Uic 
nature of the tasks assigned to students to determine 
how best to include less routine and mere engaging 
tasks, interesting tasks that carry reasonable risks. 
Considering tasks prior to developing the evalua- 
tion process shoulcl prevent Uie evaluation process 
from driving the selection and definition of tasks. 
In this way eiducators can assure an adequate repre- 
sentation of non-routine tasks in the school experi - 
ences of disadvantaged students. Some proportion 
of routine tasks might be included to offer students 
greater security in terms of being able to secure 
good evaluations for following the specified 
courses of action, particulariy when such tasks arc 
integral to mastering a certain body of knowledge. 



ERIC 



7 

13 



3. Broad Criteria Tapping Multiple Abili- 
ties, Processes, and Non-academic Do- 
mains 

Current evaluation systems have been criticized for 
focusing on overly narrow conceptions of wtiat is 
legitimate to learn in school and what Icinds of 
competencies should be assessed. Educators of 
disadvantaged students should develop broader 
criteria for the evaluation of students. Such broad- 
ening of evaluation criteria can take place in three 
ways. 

First, criteria for academic taslc performance can be 
broadened to encompass more areas of com- 
petence. It might be particularly important to de- 
velop criteria that are not closely related to reading 
ability. Second, criteria can be enlarged to include 
the processes that lead to successful task comple- 
tion as well as the flnal results of tasks. Devoting 
attention to process criteria will have both directing 
and motivating effects as students receive both bet- 
ter directions on how to approach tasks and rein- 
forcement for devoting the effort to specified pro- 
cesses. Third, evaluation criteria can include non- 
academic areas of student performance and devel- 
opment Social and affective dimensions of student 
performance often play a role in their ability to sue - 
ceed in school, and an evaluation system can both 
direct and motivate students to appropriate non-aca- 
demic behavior in ultimate support of their aca- 
demic performance. 

4. Standards that are Challenging yet 
Attainable 

Developing appropriate standards for disadvantaged 
students is not as simple as the recent school reform 
movement would suggest. It is not just a question 
of raising standards to challenge students to put 
fonh greater effort and realize greater achievement. 
There are at least three kirds of standards that must 
be considered in evaluating students: absolute stan - 
dards, relative or norm-referenced standards, and 
individual or self-referenced standards. Evaluation 
systems that primarily serve the purpose of selec- 
tion place emphasis on norm-referenced standards, 
while those that primarily serve the purpose of cer- 
tification place emphasis on absolute standards. 

Systems of evaluation designed to direct and moti- 
vate students should include self-referenced stan- 
dards which allow students to gain a sense of their 
own progress. In reality, most evaluation systems 
contain elements of each of tiiese three types of 
standards. The difficulty is in deciding how to 
combine different kinds of standards to develop an 
evaluation of students. A key to any solution to 
this situation is to understand cleariy which type of 




standard is being applied to produce any given 
assessment 

5. A Broader Range of Information and 
More Varied Collection Techniques 

The dominant techniques for collecting information 
on student performance are limited in scope and in 
the conditions under which they are employed. 
Moreover, techniques such as formal testing are 
better suited to the selection and certification pur- 
poses of evaluation than to direction and motiva- 
tion. Techniques of gathering infonnation on per- 
formance under varying conditions must be devel- 
oped to serve the direction and motivation purposes 
of evaluation. Particular atteruion must be devoted 
to making data collection experiences informative 
and engaging for disadvantaged students. 

One strategy would be to model information- 
gathering activities after real-life experiences of Uie 
students, i.e.. collect information under conditions 
similar to those in which students are likely to find 
themselves in the community or in the woricplace. 
One implication of such a sti'ategy would be to 
eliminate assessment situations that force students 
to perform within artificial time limits. Moreover, 
such information gathering should extend over a 
broader domain of performance if we hope to 
understand the nuances of the performance 
difficulties experienced by disadvantaged students. 

6. More Efficient Methods of Collecting 
Information on Student Performance 

Because collecting information on student perfor- 
mance is labor intensive, any attempt to broaden the 
range of data and the range of conditions under 
which they are coUeaed must be accompanied by 
carefid thinking about how to make the collection of 
such performance information more efficient 
Teachers have limited time for evaluation activities, 
and they must be helped to develop less labor-in- 
tensive sU'ategies for assembling information on 
performance. Another approach would be to in- 
volve teacher aides in the collection of the broader 
range of information on performance and leave the 
development of the appraisal to the teacher. 

7. Appraisals that Involve Teacher Dis- 
cretion and Broader Teacher Participation 

The dominant ways of evaluating students leave too 
litUe discretion for teacher participation. This is 
once again due to the selection and certification 
purposes of evaluation. If evaluations are to be 
compared across students or against a common 
standard, it •$ necessary to severely constrain the 
teachers' role in developing and executing the eval* 
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uation activities. However, teachers are in the best 
position to structure evaluation activities to rein- 
force the instnictional piogram and contribute to the 
direction and motivation purposes of evaluation. 
Teachers should be encouraged and tau^t how to 
develop evaluation processes that support learning 
and student developmem. Teachers should be able 
to "invent** evaluation processes that match the 
needs of their students. 

8. More Detailed Feedback 

It is clear that disadvantaged students need and 
profit from more detailed feedback on their perfor- 
mance. Detailed feedback that is related to the 
learning tasks serves not only to communicate the 
result of one perfonnanoe assessment cycle but also 
to begin the ne,it round of instruction. F^viding 
students with detailed feedback requires a great deal 
of time. Teachers should be assisted to develop 
feedback strategies that result in the most detailed 
feedback within the time constraints of their posi - 
tions. However, serious attention to detailed feed- 
back is likely to require greatei resources in terms 
of teacher time, peihaps through alterations in the 
school schedule. 

9. New Ways to Present Certification/ 
Selection Information to Students and New 
Ways to Present Direction/Motivation to 
Others 

Evaluation systems often present summary infor- 
mation on student perfbnnance to students and out- 
siders. For disadvantaged students, such informa- 
tion is often a communication of failure. Fbr out- 
siders such infomiaticm is often a communication of 
the failure of disadvanta^ students relative to less 
disadvantaged students. In each case the communi- 
cation presents problems. Disadvaiuaged students 
need to understand how they compare to others and 
to established standards, but they need to under 
stand this in the context of their own develqHnent 



Disadvantaged students can be making progress 
and still receive consistently negative infomiation 
on their standing in relation to others or established 
standards. They often lose sight of the former in 
the presentation of the latter. 

Teachere, administrators, counselors, and parents 
also need to receive information on student 
performance in broader terms than just the relative 
comparisons if they arc to move beyond the 
stereotypical assessment of disadvantaged students 
to developing creative solutions that take advantage 
of more complex patterns of student development 

10. Improvement Plans that Offer Oppor- 
tunities and Resources 

It does little good to communicate the results of an 
evaluation to students if we are not prepared to 
supply them with the resources they need to im- 
prove their performance. This is paiticulariy true 
for disadvantaged students, for whom the 
evaluative infbnnation is often negative. Peihaps 
we should adopt a rule of thumb that the oiUy 
evaluative infonnation that may be communicated is 
that which is accompanied by a plan for 
improvement complete with the appnopiim learning 
resources. Evaluations would coiKlude not with 
assessments of performance but with 
assessmentA>ppoitunity packages that are conveyed 
to students. 

Incorporating these ten elements into a system for 
evaluating student perfonnance would produce 
evaluation systems that differ dramatically from 
those presently operating in most schools. These 
elements constitute a strategy for taking more sen - 
ously the direction and motivation functions of 
evaluation systems. As such, they should lead to 
more responsive schools for all students. 
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